gpperfmon_install --enable --password $GPADMIN_PASSWORD --port $pgport --verbose and then immediately use gpconfig -c gp_enable_gpperfmon -v off. Correct?gp_enable_gpperfmon and the normal "enable" does not?gpperfmon_install if you want to use legacy. gpcc_install creates the gpperfmon_database and gpmon role if they don’t already exist. (edited) gp_enable_gpperfmon just needs to be turned off if you had installed with gpperfmon_install earlier.2019-09-17 07:44:57.864913 CEST,"gpmon","gpperfmon",p17768,th-1361094784,"10.113.96.27","18976",2019-09-17 07:44:54 CEST,0,con1519216,cm d3,seg-1,,dx54934914,,sx1,"ERROR","42P16","START equal to END for partition",,,,,,"ALTER TABLE gpmetrics.gpcc_system_history ADD PARTITION START (date '2019-10-01') INCLUSIVE END (date '2019-10-01') EXCLUSIVE WITH (APPENDONLY=true, ORIENTATION=column, COMPRESSTYPE=RLE_TYP E, COMPRESSLEVEL=1)",0,,"parse_partition.c",4115,
gpdb-2019-09-01_000000.csv:2019-09-01 17:42:02.297516 EDT,"gpmon","gpperfmon",p1917,th-1491400896,"10.193.102.203","48649",2019-08-30 17:41:59 EDT,0,con160235,cmd11547,seg-1,,dx137775,,sx1,"LOG","00000","statement: ALTER TABLE gpmetrics.gpcc_system_history ADD PARTITION START (date '2019-10-01') INCLUSIVE END (date '2019-11-01') EXCLUSIVE WITH (APPENDONLY=true, ORIENTATION=column, COMPRESSTYPE=RLE_TYPE, COMPRESSLEVEL=1)",,,,,,"ALTER TABLE gpmetrics.gpcc_system_history ADD PARTITION START (date '2019-10-01') INCLUSIVE END (date '2019-11-01') EXCLUSIVE WITH (APPENDONLY=true, ORIENTATION=column, COMPRESSTYPE=RLE_TYPE, COMPRESSLEVEL=1)",0,,"postgres.c",1590,
Pivotal changes to phpPgAdmin
- conf/config.inc.php
Change $conf['servers'][0]['desc'] from PostgreSQL to Pivotal Database
Change $conf['servers'][0]['defaultdb'] from template1 to postgres
function isGreenplum() {
gpadmin=# select oid,* from pg_tablespace;
oid | spcname | spcowner | spcacl | spcoptions
-------+------------+----------+--------+------------
1663 | pg_default | 10 | |
1664 | pg_global | 10 | |
24783 | mytblspace | 10 | |
(3 rows)
gpadmin=# SELECT * FROM gp_tablespace_location(24783);
gp_segment_id | tblspc_loc
---------------+-------------------------------
2 | /var/vcap/store/gpadmin/data2
3 | /var/vcap/store/gpadmin/data2
1 | /var/vcap/store/gpadmin/data2
0 | /var/vcap/store/gpadmin/data2
-1 | /var/vcap/store/gpadmin/data2
(5 rows)The following variables are set on the command line when the script runs. Variable Description LINK URL of the Greenplum Command Center web server. QUERYID ID of the query, if the alert was triggered by a query. SERVERNAME Name of the Greenplum Command Center server. QUERYTEXT The text of the query, if the alert was triggered by a query. ACTIVERULENAME Current text the of rule, with user-specified values included. LOGID Value of this alert's id column in the gpmetrics.gpcc_alert_log table. RULEDESCRIPTION Text of the rule, including user-specified values, at the time the alert was raised. ALERTDATE Date the alert was raised. ALERTTIME Time the alert was raised. SUBJECT Subject line for email.
LINK=… QUERYID=…,SERVERNAME=...QUERYTEXT=… $MASTER_DATA_DIRECTORY/gpmetrics/send_alert.sh for ARGUMENT in "$@"
do
KEY=$(echo "${ARGUMENT}" | cut -f1 -d=)
VALUE=$(echo "${ARGUMENT}" | cut -f2- -d=)
--password gpmon_password and doesn't prompt for a password like gpccinstall-x-x-x does. By prompting for a password, it defeats the purpose of the -auto feature.[root@ddlgpmtst11a logrotate.d]# cat gpcc
/usr/local/greenplum-cc-web/logs/* {misingok
minsize 100M
daily
gpstop -u should be added to the end of the gpmon role Authentication section of https://gpcc.docs.pivotal.io/600/topics/setup-collection-agents.html . gpcc not being able to start.[gpadmin@pivotal-gpdb-6 ~]$ gpconfig -s shared_preload_libraries Values on all segments are consistent GUC : shared_preload_libraries Master value: metrics_collector Segment value: metrics_collector

3 years agopsql -d testdb -f $GPHOME/share/postgresql/contrib/gp_session_state.sql
Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sdb1 104804356 5161320 99643036 5% /data
ExecInitNode that normally does gpmon packet initialization. As a result gpmon is not initialized for all dynamic sidecar nodes that causes bad magic 0 warnings. Here is an example to reproduce a problem (gp_enable_gpperfmon should be on):create table t(a int, b date, c int) distributed by (a)
partition by range (b) subpartition by list (c)
subpartition template
(subpartition s1 values (1), subpartition s2 values(2))
(start ('2020-04-01') end('2020-05-01') every(interval '1 day'));
ps if this process is running on master and each segment host.- in the instance_name in the config file but it doesn't alert you to this which then causes the above error message and failure to create an instance.2020-07-31 13:33:24|:-LOG: Making initial connection to mdw 2020-07-31 13:33:40|:-LOG: Connection to mdw lost. Restarting gpsmon. 2020-07-31 13:33:57|:-LOG: Connection to mdw lost. Restarting gpsmon. 2020-07-31 13:34:13|:-LOG: Connection to mdw lost. Restarting gpsmon.
[gpadmin@mdw logs]$ cat /data3/primary/gpseg2/gpperfmon/gpsmon.2020.07.31_175021.log 2020-07-31 13:50:21|:-LOG: HOSTNAME = 'mdw' 2020-07-31 13:50:21|:-FATAL: [INTERNAL ERROR gpsmon.c:2152] unable to bind udp socket error 98 (Address already in use) ... exiting
[gpadmin@mdw ~]$ netstat -nap | grep 8888 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp6 0 0 :::8888 :::* LISTEN 26425/gpsmon udp6 0 0 :::8888 :::* 26425/gpsmon
gpadmin 27084 27075 0 16:15 ? 00:00:00 /usr/local/greenplum-db-4.3.33.7/bin/gpmmon -D /data1/master/gpseg-1/gpperfmon/conf/gpperfmon.conf -p 6432/data1/master/gpseg-1/gpperfmon/conf/gpperfmon.conf doesn't exist?2020/08/31 13:40:50.239 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 1001 (going away) 2020/08/31 13:41:24.496 [I] [websocket.go:85] WebsocketController register user mkanchwa map[basic:false operator:true operator_basic:false superuser:true user:false] 2020/08/31 13:41:24.521 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 1001 (going away) 2020/08/31 13:41:40.107 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 1001 (going away) 2020/08/31 13:41:40.625 [I] [websocket.go:85] WebsocketController register user mkanchwa map[basic:false operator:true operator_basic:false superuser:true user:false] 2020/08/31 13:41:40.798 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 4000: by gpcc application
query blocked for 5min,blocked query: TRUNCATE TABLE ODS_MES_LOT,blocked query
--password switch to the -auto option like we had with gpperfmon_install?SELECT t.sample_time,
COALESCE(swap_page_in, 0) AS swap_page_in,
COALESCE(swap_page_out, 0) AS swap_page_out,
COALESCE(cpu_user, 0) AS cpu_user,
COALESCE(cpu_sys, 0) AS cpu_sys,
-rwxrwxr-x. 1 gpadmin gpadmin 683584 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.9.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683628 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.9.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683519 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.8.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683607 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.8.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683469 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.7.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683496 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.7.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683497 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.6.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683513 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.5.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683384 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.4.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 681616 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.3.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 681427 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.2.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683890 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.11.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683907 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.11.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 681486 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.1.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683887 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.10.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 683898 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.10.0-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 681197 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.0.1-rhel7-x86_64.gppkg -rwxrwxr-x. 1 gpadmin gpadmin 681424 Sep 25 12:29 MetricsCollector-6.3.1_gp_6.0.0-rhel7-x86_64.gppkg
pg_logdir_ls SQL built-in function in Greenplum is used by gpcc or any other monitoring tool that this team owns. I couldn’t find it being used in Greenplum repository. We are assessing impact of changing or removing this function altogether.below command center directory is getting full, which is causing /usr directory to be full on master Greenplum server. /usr/local/greenplum-cmd-ctr/greenplum-cc-web-4.6.1/ccdata -rw-r--r-- 1 gpadmin gpadmin 3667156 Nov 2 12:27 gpcc_queryinfo.dat2020-11-02_122708 -rw-r--r-- 1 gpadmin gpadmin 508 Nov 2 12:27 gpcc_queryinfo.dat.ext2020-11-02_122708 -rw-rw-r-- 1 gpadmin gpadmin 8433 Nov 2 12:42 gpcc_system_2020-11-02T12:42:42.53287853-05:00.dat -rw-r--r-- 1 gpadmin gpadmin 1368153 Nov 2 12:42 gpcc_queryinfo.dat2020-11-02_124251 -rw-rw-r-- 1 gpadmin gpadmin 2069 Nov 3 22:50 catalog -rw-r--r-- 1 gpadmin gpadmin 953 Nov 4 20:16 gpcc_queryinfo.dat.ext -rw-r--r-- 1 gpadmin gpadmin 8896577536 Nov 4 20:16 gpcc_queryinfo.dat Is there any rotation setting to manage file - gpcc_queryinfo.dat which is shown occupying 8.3 GB of space ? Also, do we need to change any parameters for rotation of other files in this directory?
# quantum specifies the time in seconds between updates from # performance monitor agents on all segments. Valid values # are 10, 15, 20, 30, or 60 quantum = 15 # min_query_time specifies the minimum query run time # in seconds for statistics collection. The monitor logs all # queries that run longer than this value in the queries_history # table. For queries with shorter run times, no historical # data is collected. min_query_time = 20 # This should be a percentage between 0 and 100 and should be # less than the error_disk_space_percentage. If a filesystem's # disk space used percentage equals or exceeds this value a # warning will be logged and a warning email/snmp trap may be # sent. If this configuration is set to 0 or not specified, no # warnings are sent. #warning_disk_space_percentage = 80 # This should be a percentage between 0 and 100 and should be # greater than the warning_disk_space_percentage. If a # filesystem's disk space used percentage equals or exceeds # this value an error will be logged and a error email/snmp # trap may be sent. If this configuration is set to 0 or not # specified, no errors are sent. #error_disk_space_percentage = 90 #This is the interval in minutes that limits the number of #error/warning messages that are sent. The minimum value for #this configuration is 1. Setting this to 0 or not specifying #this configuration results in it getting set to the minimum. disk_space_interval = 60 #This is the maximum number of error/warning messages that #will be sent in the disk_space_interval. The maximum value #for this configuration is 50. The minimum value for this #configuration is 1. Setting this configuration to greater #than 50 or not specifying this configuration results in it #getting set to the maximum. max_disk_space_messages_per_interval = 10 # The number of partitions for statistics data in month # will be retained. Older partitions will be dropped. #partition_age = 6 log_location = /data1/master/gpseg-1/gpperfmon/logs
[gpadmin@dcmipphpgp030 ~]$ gpconfig -s gp_email_smtp_server 20201110:10:33:27:111505 gpconfig:dcmipphpgp030:gpadmin-[ERROR]:-Failed to retrieve GUC information, guc does not exist: gp_email_smtp_server 'NoneType' object is not iterable [gpadmin@dcmipphpgp030 ~]$ gpconfig -s gp_email_smtp_userid 20201110:10:34:14:112063 gpconfig:dcmipphpgp030:gpadmin-[ERROR]:-Failed to retrieve GUC information, guc does not exist: gp_email_smtp_userid 'NoneType' object is not iterable [gpadmin@dcmipphpgp030 ~]$ gpconfig -s gp_email_smtp_password 20201110:10:34:19:112125 gpconfig:dcmipphpgp030:gpadmin-[ERROR]:-Failed to retrieve GUC information, guc does not exist: gp_email_smtp_password 'NoneType' object is not iterable
[alert] emailFrom = noreply@abc.com smtpServer = smtp.gmail.com:587 smtpUsername = username@gmail.com smtpPassword = password emailTo = smtpEncryption = insecure smtpAuthentication = auto
2020-12-04 05:00:08.052014 EST,"gp_powerbi_cashflowdashboards","maxisdb",p12606,th-1847465856,"10.126.187.30","54851",2020-12-04 05:00:07 EST,0,con55053,cmd6,seg-1,,,,sx1,"ERROR","58M01","failed to acquire resources on one or more segments","FATAL: out of memory
(seg29 10.125.173.114:16004)",,,,,"SELECT
REV.VEH_ID_NBR AS ""VIN""
, REV.INV_EVNT_DT AS ""Invoice Event Date""
, REV.INV_ID AS ""Identifier""
, REV.LAST_INV_FLAG AS ""Last Invoice Flag""
, REV.VEH_INV_ORD_TYPE_CD AS ""Vehicle Invoice Order Type Code""
, REV.DW_MOD_TS AS ""Last Processed Time""
, MAX(REV.INV_CRT_DT) AS ""Invoice Date""
FROM FINC.VEH_WHLSL_INV_EVENT_M REV
INNER JOIN FINC.FS_VEH_IDENT GVIT
ON REV.VEH_ID_NBR = GVIT.VEH_ID_NBR
INNER JOIN FINC.INCNTV_PYMT_F INCNTV
ON REV.VEH_ID_NBR = INCNTV.VEH_ID_NBR
WHERE LAST_INV_FLAG <> 'N'
AND extract(year from INCNTV_PYMT_DT) >= extract(year from current_date)-2
--AND GVIT.VIN_MODEL_YR_NBR >= YEAR(CURRENT_DATE) - 2
AND UPPER(GVIT.UMF_XREF_FINC_MKTG_DIV) IN ('CHEVROLET','GMC','BUICK','CADILLAC','HOLDEN')
GROUP BY 1,2,3,4,5,6",0,,"cdbgang_async.c",219,2020-12-04 05:00:07.501120 EST,"gp_powerbi_cashflowdashboards","maxisdb",p39537,th342636672,"10.125.173.107","60080",2020-12-04 05:00:07 EST,0,con55053,,seg29,,,,sx1,"FATAL","53200","out of memory",,,,,,,0,,"procarray.c",2033, ... ... ... 2020-12-04 05:00:50.371721 EST,"gpadmin","maxisdb",p1080,th342636672,"10.125.173.107","24450",2020-12-04 05:00:50 EST,0,,,seg29,,,,sx1,"FATAL","53200","Out of memory. Failed on request of size 1720 bytes. (context 'CacheMemoryContext') (aset.c:1376)",,,,,,,0,,"mcxt.c",568,
[gpadmin@dcmipphpgp007 pg_log]$ gpconfig -s runaway_detector_activation_percent Values on all segments are consistent GUC : runaway_detector_activation_percent Master value: 90 Segment value: 90 [gpadmin@dcmipphpgp007 pg_log]$ ========================== [gpadmin@dcmipphpgp007 ~]$ cat gp_resgroup_config.out groupid | groupname | concurrency | cpu_rate_limit | memory_limit | memory_shared_quota | memory_spill_ratio | memory_auditor | cpuset ---------+-----------------+-------------+----------------+--------------+---------------------+--------------------+----------------+-------- 5026154 | metadata_group | 10 | -1 | 10 | 80 | 0 | vmtracker | 7 2720032 | spark_group | 60 | 10 | 10 | 80 | 0 | vmtracker | -1 16407 | gg_group | 40 | 10 | 5 | 80 | 0 | vmtracker | -1 6437 | default_group | 10 | 1 | 0 | 80 | 0 | vmtracker | -1 4334202 | adhoc_group | 20 | 15 | 5 | 80 | 0 | vmtracker | -1 16423 | etl_group | 20 | 20 | 20 | 80 | 0 | vmtracker | -1 16415 | reporting_group | 30 | 25 | 30 | 80 | 0 | vmtracker | -1 2720024 | dba_group | 10 | 5 | 5 | 80 | 0 | vmtracker | -1 6438 | admin_group | 10 | 9 | 10 | 80 | 0 | vmtracker | -1 (9 rows)
runaway_detector_activation_percent sets the percent of utilized global shared memory that triggers the termination of queries that are managed by resource groups that are configured to use the vmtracker memory auditor, such as admin_group and default_group. Resource groups have a global shared memory pool when the sum of the MEMORY_LIMIT attribute values configured for all resource groups is less than 100. For example, if you have 3 resource groups configured with MEMORY_LIMIT values of 10 , 20, and 30, then global shared memory is 40% = 100% - (10% + 20% + 30%).
Greenplum Database uses resource group memory more efficiently when you leave some memory (for example, 10-20%) unallocated for the global shared memory pool. The availability of global shared memory also helps to mitigate the failure of memory-consuming or unpredicted queries.
2 years ago
2 years ago
2 years agogpcc stop and gpcc startsudo tcpdump -nvvv -A -i any port 28080 -w gpcc.pcap
gpcc --version shows 6.4. Tried new browser, cleared cache…any thoughts here? I could just re-install but wanted to point this out. thanksv5.17-> v5.28.5 . From the gpdb master log, there are quite some gpcc related duration trace like:...
2021-01-23 09:48:30.221171 HKT,"gpmon","gpperfmon",p47028,th-1601132672,"[local]",,2021-01-23 09:38:11 HKT,0,con26,cmd13,seg-1,slice1,,,,"LOG","00000","duration: 619180.189 ms statement: INSERT INTO gpmetrics.gpcc_pg_log_history SELECT * FROM gpmetrics._gpcc_pg_log_import WHERE logseverity IN ('WARNING', 'ERROR', 'FATAL', 'PANIC')",,,,,,"INSERT INTO gpmetrics.gpcc_pg_log_history SELECT * FROM gpmetrics._gpcc_pg_log_import WHERE logseverity IN ('WARNING', 'ERROR', 'FATAL', 'PANIC')",0,,"postgres.c",1406,
2021-01-23 09:48:30.258591 HKT,"gpmon","gpperfmon",p46938,th-1601132672,"10.19.37.199","29023",2021-01-23 09:38:10 HKT,21826088,con26,cmd13,seg-1,,dx149,x21826088,sx1,"LOG","00000","duration: 619227.586 ms statement: INSERT INTO gpmetrics.gpcc_pg_log_history SELECT * FROM gpmetrics._gpcc_pg_log_import WHERE logseverity IN ('WARNING', 'ERROR', 'FATAL', 'PANIC')",,,,,,"INSERT INTO gpmetrics.gpcc_pg_log_history SELECT * FROM gpmetrics._gpcc_pg_log_import WHERE logseverity IN ('WARNING', 'ERROR', 'FATAL', 'PANIC')",0,,"postgres.c",1862,
2021-01-23 09:52:24.852070 HKT,"gpmon","odsplus",p1563,th-1601132672,"10.19.37.199","52959",2021-01-23 09:49:00 HKT,21826239,con129,cmd8,seg-1,,dx1774,x21826239,sx1,"LOG","00000","duration: 204068.551 ms statement: SELECT pg_temp.gpcc_estimate_live_dead_tuples(c.oid) AS tps, c.oid AS oid, 1 AS is_seg
...
4:08 AM
).2021/06/11 08:17:29.967 [I] [logon.go:64] User attempted login: jz967l 2021/06/11 08:17:30.291 [I] [logon.go:110] Authenticated user (jz967l) ok, superuser(true), operator(true), operator_basic(false), basic(false) 2021/06/11 08:17:31.143 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 4500: by gpcc application 2021/06/11 08:17:31.583 [I] [websocket.go:85] WebsocketController register user jz967l map[basic:false operator:true operator_basic:false superuser:true user:false] 2021/06/11 08:37:53.983 [I] [websocket.go:85] WebsocketController register user map[] 2021/06/11 08:55:17.603 [W] [websocket.go:151] WebsocketController cannot read message: websocket: close 1006 (abnormal closure): unexpected EOF 2021/06/11 08:56:44.034 [I] [websocket.go:85] WebsocketController register user jz967l map[basic:false operator:true operator_basic:false superuser:true user:false]
gpperfmon=# select max(logtime) from gpmetrics.gpcc_pg_log_history;
max
----------------------------
2020-07-14 17:39:29.659221
(1 row)
gpperfmon=# select * from pg_stat_last_operation where objid='gpmetrics.gpcc_pg_log_history'::regclass;
classid | objid | staactionname | stasysid | stausename | stasubtype | statime
---------+-------+---------------+----------+------------+------------+-------------------------------
1259 | 71258 | CREATE | 10 | gpadmin | TABLE | 2020-01-13 10:29:21.771723-05
1259 | 71258 | ALTER | 10 | gpadmin | OWNER | 2020-01-13 11:19:37.238397-05
1259 | 71258 | ANALYZE | 10 | gpadmin | | 2020-10-22 03:58:49.998781-04
1259 | 71258 | PARTITION | 17167 | gpmon | SPLIT | 2021-06-01 08:59:16.084847-04
1259 | 71258 | PRIVILEGE | 17173 | jz967l | GRANT | 2021-04-09 19:25:38.064775-04
(5 rows)We have GPCC 4.12 installed in our system. We have set WLM rules in that through the GPCC GUI. We would like to set these rules as scripts so that these rules will only apply during certain times. Could you please let us know how do we run these rules as a script? I want to set the following rules as script. Can you please provide the approximate and example script for that? If resource group = default_group AND query running time is greater than 1h If resource group = default_group AND slice is greater than 200 If resource group = default_group AND total disk IO is greater than 1TB
Unexpected internal error: Master process received signal SIGBUS2021-09-27 02:30:01.731383 AEST,"gpmon","gpperfmon",p608621,th-1917896832,"10.138.76.40","11309",2021-09-27 02:25:17 AEST,73593092,con51954727,cmd4752,seg-1,,dx7476417,x73593092,sx1,"FATAL","57P01","terminating connection due to administrator command",,,,,,"update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-47496917-23.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-47496917-23.txt')), E'\n')) where tmid = 1630864202 and ssid = 47496917 and ccnt = 23 and ctime >= now() - interval '2 months'",0,,"postgres.c",3670,
2021-09-27 02:30:01.776000 AEST,,,p608621,th0,,,2021-09-27 02:25:17 AEST,0,con51954727,cmd4752,seg-1,,,,,"PANIC","XX000","Unexpected internal error: Master process received signal SIGBUS",,,,,,,0,,,2021-09-27 02:29:59.223470 AEST,"gpmon","gpperfmon",p608621,th-1917896832,"10.138.76.40","11309",2021-09-27 02:25:17 AEST,0,con51954727,cmd4712,seg-1,,dx7476352,,sx1,"LOG","00000","2021-09-27 02:29:59:223402 AEST,THD000,NOTICE,""Feature not supported: SIRV functions"",",,,,,,"update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1087.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1087.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1087 and ctime >= now() - interval '2 months'",0,,"COptTasks.cpp",527,
2021-09-27 02:29:59.325886 AEST,"gpmon","gpperfmon",p608621,th-1917896832,"10.138.76.40","11309",2021-09-27 02:25:17 AEST,0,con51954727,cmd4714,seg-1,,dx7476354,,sx1,"LOG","00000","statement: update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1089 and ctime >= now() - interval '2 months'",,,,,,"update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1089 and ctime >= now() - interval '2 months'",0,,"postgres.c",1590,
2021-09-27 02:29:59.342400 AEST,"gpmon","gpperfmon",p608621,th-1917896832,"10.138.76.40","11309",2021-09-27 02:25:17 AEST,0,con51954727,cmd4714,seg-1,,dx7476354,,sx1,"LOG","00000","2021-09-27 02:29:59:342337 AEST,THD000,NOTICE,""Feature not supported: SIRV functions"",",,,,,,"update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1089.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1089 and ctime >= now() - interval '2 months'",0,,"COptTasks.cpp",527,
2021-09-27 02:29:59.441948 AEST,"gpmon","gpperfmon",p608621,th-1917896832,"10.138.76.40","11309",2021-09-27 02:25:17 AEST,0,con51954727,cmd4716,seg-1,,dx7476356,,sx1,"LOG","00000","statement: update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1091.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1091.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1091 and ctime >= now() - interval '2 months'",,,,,,"update gpmetrics.gpcc_queries_history set query_text = rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1091.txt')), E'\n'), query_hash = md5(rtrim((select gpcc_schema.read_file('/data/master/gpseg-1/gpmetrics/query_text/q1630864202-49350609-1091.txt')), E'\n')) where tmid = 1630864202 and ssid = 49350609 and ccnt = 1091 and ctime >= now() - interval '2 months'",0,,"postgres.c",1590,
allow_trust_logon in GPCC? I can not see the “comment out” line in the app.conf, and I tried to added one and set to yes (also tried true) but still can not login with trusted user (already restarted gpcc). I also tried to search the keyword allow_trust_logon in gpcc code but seem I can not get anything. so I guess this settings already deprecated right?gpmetrics.gpcc_resgroup_history, is a new table that stores the history of the resource consumption of each resource group on each segment
A user can now now determine the TLS cipher suites that Command Center accepts. For more information, see the Command Center Console Parameters documentation page.
1 year ago
(edited) /data/master/pedw_gpseg-xxxxxxrics/query_text/qxxxxxxxxxxxxxxxxxxxxxxxxt pq: out of memory
plpy.Error: file /data/master/pedw_gpseg-1/gpmetrics/query_text/q1642314941-5761931-153190.txt unavailable (plpy_elog.c:121) u:gpmondb:gpperfmon
memory_info and dynamic_memory_info such as https://gpdb.docs.pivotal.io/6-20/ref_guide/gpperfmon/db-memory-info.htmlcreate table t1 (a int) ; insert into t1 values (generate_series(1,10000000)); select * from t1 a join t1 b on a.a=b.a join t1 c on b.a=c.a join t1 d on d.a=c.a order by 1 desc;
2022-05-25 12:14:21.594343 CEST,,,p3241359,th685318784,,,,0,,,seg12,,,,,"LOG","00000","Metrics collector: socket_init failed 111",,,,,,,0,,"socket_utils.c",393,
The following cookie was issued by the application and does not have the secure flag set: beegosessionID The cookie appears to contain a session token, which may increase the risk associated with this issue. You should review the contents of the cookie to determine its function.
2 instances of this issue were identified, at the following locations: /assets/main.js /static/assets/main.js The library YUI version 2.9.0 has known security issues. The vulnerability is affecting all versions prior 2.9.1 (between 2.8.0 and 2.9.1) The vulnerability might be affecting a feature of the library that the website is not using. If the vulnerable feature is not used, this alert can be considered false positive. The library name and its version are identified based on a Retire.js signature. If the library identification is not correct, the prior vulnerability does not apply. Recommendation Upgrade to a secure version
Issue detail The Strict-Transport-Security header is missing the "includeSubDomains" parameter. This ensures that all the site's subdomain are also protected with HSTS. The application potentially fails to prevent users from connecting to some of the site's subdomains over unencrypted connections. An attacker able to modify a legitimate user's network traffic could bypass the application's use of SSL/TLS encryption, and use the application as a platform for attacks against its users. This attack is performed by rewriting HTTPS links as HTTP, so that if a targeted user follows a link to the site from an HTTP page, their browser never attempts to use an encrypted connection. The sslstrip tool automates this process. To exploit this vulnerability, an attacker must be suitably positioned to intercept and modify the victim's network traffic. This scenario typically occurs when a client communicates with the server over an insecure connection such as public Wi-Fi, or a corporate or home network that is shared with a compromised computer. Common defenses such as switched networks are not sufficient to prevent this. An attacker situated in the user's ISP or the application's hosting infrastructure could also perform this attack. Note that an advanced adversary could potentially target any connection made over the Internet's core infrastructure. Recommendation Include the includeSubDomains parameter in the Strict-Transport-Security header.
Issue detail 1 instance of this issue was identified, at the following location: /static/assets/%7B%7B.JSPath%7D%7D If a page fails to set an appropriate X-Frame-Options or Content-Security-Policy HTTP header, it might be possible for a page controlled by an attacker to load it within an iframe. This may enable a clickjacking attack, in which the attacker's page overlays the target application's interface with a different interface provided by the attacker. By inducing victim users to perform actions such as mouse clicks and keystrokes, the attacker can cause them to unwittingly carry out actions within the application that is being targeted. This technique allows the attacker to circumvent defenses against cross-site request forgery and may result in unauthorized actions. Note that some applications attempt to prevent these attacks from within the HTML page itself, using "framebusting" code. However, this type of defense is normally ineffective and can usually be circumvented by a skilled attacker. You should determine whether any functions accessible within frameable pages can be used by application users to perform any sensitive actions within the application. Recommendation To effectively prevent framing attacks, the application should return a response header with the name X-Frame-Options and the value DENY to prevent framing altogether, or the value SAMEORIGIN to allow framing only by pages on the same origin as the response itself. Note that the SAMEORIGIN header can be partially bypassed if the application itself can be made to frame untrusted websites.
gpcc.enable_send_inner_query
[gpadmin@mdw-lab1 greenplum-cc-4.15.0]$ gpconfig -l |grep gpcc [name: gpcc.enable_query_profiling] [unit: None] [context: user] [vartype: bool] [min_val: None] [max_val: None] [name: gpcc.enable_send_query_info] [unit: None] [context: user] [vartype: bool] [min_val: None] [max_val: None] [name: gpcc.metrics_collector_pkt_version] [unit: ] [context: internal] [vartype: integer] [min_val: 0] [max_val: 65535] [name: gpcc.query_metrics_port] [unit: ] [context: user] [vartype: integer] [min_val: 1024] [max_val: 65535] [name: gpcc.query_tags] [unit: None] [context: user] [vartype: string] [min_val: None] [max_val: None] [gpadmin@mdw-lab1 greenplum-cc-4.15.0]$ gpadmin=# set gpcc. gpcc.enable_query_profiling gpcc.query_metrics_port gpcc.query_tags gpcc.enable_send_query_info gpadmin=# \q [gpadmin@mdw-lab1 greenplum-cc-4.15.0]$ gpconfig -s shared_preload_libraries Values on all segments are consistent GUC : shared_preload_libraries Master value: metrics_collector,gp_wlm Segment value: metrics_collector,gp_wlm [gpadmin@mdw-lab1 greenplum-cc-4.15.0]$
Can you confirm that GPCC is not using the Flash component which is the vulnerable piece of YUI 2.9.0, according to CVE-2012-5883 https://www.cvedetails.com/cve/CVE-2012-5883/
gpmetrics.gpcc_queries_history and not finding it?CREATE VIEW public.alation_qli_view AS
SELECT
ctime,
rows_out,
tfinish,
tstart,
cpu_elapsed,
status,
cast (tmid as varchar(10)),
ssid,
ccnt,
username,
application_name,
query_text,
tsubmit,
db,
tmid as pid
FROM queries_history
WHERE
lower(username) NOT IN ('gpmon')
AND query_text NOT ILIKE '%jdbc_savepoint%'
AND query_text NOT ILIKE 'select current_schema()';
9 months agoGreenplum Command Center 6.8 is compatible with the following platforms. Tanzu Greenplum Database 6.x
"GPCC [version 6.8.2] may not be fully compatible with GPDB [6.21.2]. Please refer to the version compatibility in the GPCC documentation."
9 months ago
9 months ago
customer confirmed that they have done this must stop to install the metric collector. But still see the error message Error: GPCC [version 6.8.2] may not be fully compatible with GPDB [6.21.2]. Please refer to the version compatibility in the GPCC documentation.
9 months ago20221010:09:57:35:003192 gppkg:smdw:gpadmin-[INFO]:-Starting gppkg with args: -q --all pivotal_greenplum_backup_restore-1.24.0 postgis-2.5.4+pivotal.7.build.1 MetricsCollector-6.8.2_gp_6.21.2
8 months agogpmetrics.gpcc_resgroup_history table that feeds into GPCC to get a historical picture of the % usage of the Resource Group cpu_rate_limits.Their end goal is to be able to decide whether to decrease the cpu_rate_limit for RGs.gpmetrics.gpcc_resgroup_history is documented here.cpu_usage_percent is documented as “the sum of the percentages of CPU cores that are used by the resource group”, so is it correct to say that cpu_usage_percent is analogous to output from an uptime?I ask, as their gpmetrics.gpcc_resgroup_history table shows that cpu_usage_percent far exceeds cpu_rate_limit regularly:gpperfmon=# select ctime,cpu_rate_limit,cpu_usage_percent from gpmetrics.gpcc_resgroup_history where cpu_usage_percent > cpu_rate_limit AND segid=0 limit 5;
ctime | cpu_rate_limit | cpu_usage_percent
---------------------+----------------+-------------------
2022-10-24 10:10:30 | 15 | 18.68
2022-10-24 10:10:45 | 15 | 23.36
2022-10-24 10:11:30 | 15 | 20.1
2022-10-24 10:14:30 | 15 | 42.09
2022-10-24 10:16:15 | 15 | 31.13
(5 rows)
gpperfmon=# Why is CPU usage for the resource group higher than the configured CPU_RATE_LIMIT? This situation can occur in the following circumstances: A resource group may utilize more CPU than its CPU_RATE_LIMIT when other resource groups are idle. In this situation, Greenplum Database allocates the CPU resource of an idle resource group to a busier one. This resource group feature is called CPU burst. The operating system CPU scheduler may cause CPU usage to spike, then drop down. If you believe this might be occurring, calculate the average CPU usage within a given period of time (for example, 5 seconds) and use that average to determine if CPU usage is higher than the configured limit.
c:\Program Files\MIT\Kerberos\bin>kinit.exe -k -t C:\Users\IEUser\Desktop\gpdb-k erberos.keytab gpmon@GPDB.KRB [1644] 1667856633.926001: Getting initial credentials for gpmon@GPDB.KRB [1644] 1667856633.926002: Looked up etypes in keytab: aes256-cts, aes128-cts [1644] 1667856633.926004: Sending request (178 bytes) to GPDB.KRB [1644] 1667856633.926005: Resolving hostname gpdb-kdc [1644] 1667856633.926006: Sending initial UDP request to dgram 10.225.28.79:88 [1644] 1667856634.175000: Received answer (649 bytes) from dgram 10.225.28.79:88 [1644] 1667856634.175001: Response was not from master KDC [1644] 1667856634.175002: Processing preauth types: 19 [1644] 1667856634.175003: Selected etype info: etype aes128-cts, salt "GPDB.KRBg pmon", params "" [1644] 1667856634.175004: Produced preauth for next request: (empty) [1644] 1667856634.175005: Getting AS key, salt "GPDB.KRBgpmon", params "" [1644] 1667856634.175006: Retrieving gpmon@GPDB.KRB from FILE:C:\Users\IEUser\De sktop\gpdb-kerberos.keytab (vno 0, enctype aes128-cts) with result: 0/Success [1644] 1667856634.175007: AS key obtained from gak_fct: aes128-cts/AA10 [1644] 1667856634.175008: Decrypted AS reply; session key is: aes128-cts/18FC [1644] 1667856634.175009: FAST negotiation: available [1644] 1667856634.175010: Initializing API:Initial default ccache with default p rinc gpmon@GPDB.KRB [1644] 1667856634.175011: Storing gpmon@GPDB.KRB -> krbtgt/GPDB.KRB@GPDB.KRB in API:Initial default ccache [1644] 1667856634.175012: Storing config in API:Initial default ccache for krbtg t/GPDB.KRB@GPDB.KRB: fast_avail: yes [1644] 1667856634.175013: Storing gpmon@GPDB.KRB -> krb5_ccache_conf_data/fast_a vail/krbtgt\/GPDB.KRB\@GPDB.KRB@X-CACHECONF: in API:Initial default ccache c:\Program Files\MIT\Kerberos\bin>klist.exe Ticket cache: API:Initial default ccache Default principal: gpmon@GPDB.KRB Valid starting Expires Service principal 11/07/22 13:30:33 11/08/22 13:30:33 krbtgt/GPDB.KRB@GPDB.KRB c:\Program Files\MIT\Kerberos\bin>
6 months ago
6 months agogpccws is consuming too much memory this morning, ~ 90% of the master memory, they are on GPCC6.7.2, and they mentioned that the system load was not high at that time.. wondering shall I create a jira for this or R&D can quickly identify any known issue? They have prof data provided (edited) dmesg --time-format=ctimeIf dmesg is not available, also consider running sudo fsck -n <> on the mount point to check for errors."A HTTP request is sent by the scanner to the target being scanned with a benign payload containing a unique token. The target, if vulnerable, will act on the payload. Tenable tracks the target's action on the payload via a callback to our hosted environment …. based on the unique token that was embedded in the initial request or via the LDAP connection callback to the scanner for plugin 155998." In a Splunk search we're able to see these servers making that query out to the Nessus servers, but not reaching our scanner:
The reason we raised this ticket is that GPCC team has advised us to use gpcc 6.7.2 with 6.23.0 when we downgraded GPCC from 6.8 to 6.7.0 back and queries history stopped populating post downgrade. We want a confirmation from you that on a fresh install (no downgrades involved) that 6.7.0 works as expected with 6.23.0 and if only performing downgrades will break the compatibility. PS : We have a QA cluster with fresh install of 6.23.0 and 6.7.0 working without issue.
2023-04-18 11:02:20 Starting the gpcc agents and webserver... 2023-04-18 11:02:54 Failed to detect agent start status, please refer /var/gpadmin/greenplum-cc/logs for details
idb2529):2023-04-18 11:02:25 [ERROR]ControlCommand: idb2529 error when receive from grpc stream: rpc error: code = Canceled desc = context canceled 2023-04-18 11:02:25 [ERROR]ControlCommand: idb2529 error when receive from grpc stream: rpc error: code = Canceled desc = context canceled 2023-04-18 11:02:25 [INFO]ControlCommand: idb2529 stream disconnected 2023-04-18 11:02:25 [INFO]ControlCommand: idb2529 receiver routine exit 2023-04-18 11:02:25 [INFO]ControlCommand: idb2529 sender routine exit
3:13 PM